A FISTA-type accelerated gradient algorithm for solving smooth nonconvex composite optimization problems

In this paper, we describe and establish iteration-complexity of two accelerated composite gradient (ACG) variants to solve a smooth nonconvex optimization problem whose objective function is the sum differentiable f with Lipschitz continuous simple nonsmooth closed convex h. When convex, first ACG variant reduces well-known FISTA for specific choice input, hence one can be viewed as natural extension latter setting. The requires an input pair (M, m) such that m-weakly $$\nabla f$$ M-Lipschitz continuous, $$m \le M$$ (possibly $$m<M$$ ), which usually hard obtain or poorly estimated. second on other hand start from arbitrary positive scalars its complexity shown not worse, better in some cases, than large range pairs. Finally, numerical results are provided illustrate efficiency variants.